Graphics Plus/Graphics Plus.iso/general/hdf/docs/hdfvset.lha/HDFVset.ch1

Text File | 1994-01-10 | 24KB | 694 lines

1.1    NCSA HDF Vset


NCSA HDF Vset Basics    1.1


National Center for Supercomputing Applications

November 1990







1.1    NCSA HDF Vset


NCSA HDF Vset Basics    1.1


National Center for Supercomputing Applications

November 1990










Chapter 1    NCSA HDF Vset Basics



Chapter Overview
Understanding NCSA HDF Vset
Deciding on a Vset Scheme
Examining the HDF Vset Interface
Using Examples
Example 1:  Writing Data
Example 2:  Reading Data
Example 3:  Using Meshes and Connectivity
        Lists
Example 4:  Linking Vset Elements
Using the HDF Vset Routines
Naming Vgroups and Vdatas

Chapter Overview

This chapter introduces the NCSA HDF Vset and presents several
examples on reading and writing data using the vset interface.


Understanding NCSA HDF Vset

An HDF Vset is a logical grouping of diverse, but related data
items within an HDF file. Data organization within the file
resembles the UNIX file system. Only two basic types of storage
elements exist in a vset:  vgroups and vdatas. A vgroup is like a
directory or subdirectory, while a vdata is like a file. Thus, a vdata
may only contain data, while a vgroup only contains references to
vdatas or to other vgroups. The entire vset may be viewed as a
directory tree with vgroups as nodes and vdatas as leaves. Figure
1.1 shows a vset with data stored in five vdatas while Figure 1.2
shows two vsets in an open file. Note that several vgroups may
point to the same vgroup or vdata.

Vdatas only store data. The data is organized into fields within
each vdata. Each field is identified by a unique fieldname. The
type of each field may be either integer, character or float. Fields of
different types may exist within a vdata.

Figure 1.1    Analogy Between the HDF Vset and the UNIX File System



Figure 1.2    Sharing Data Among Vsets




Deciding on a Vset Scheme

The HDF Vset is a versatile hierarchical storage scheme for
grouping simple or multi-dimensional datasets. Multi-variable
datasets are conveniently stored as vsets by first identifying the
fields in the datasets.

As many fields as needed may be defined for storing data in a
vset. There are many ways of organizing these fields. At one
extreme, all the fields are organized in one huge vdata (vset C in
Figure 1.3). At the other extreme, data for one field is stored in one
vdata (vset B in Figure 1.3). Usually, closely related fields are
organized together in one vdata (vset A in Figure 1.3).

After all the fields are organized into vdatas, vgroups should
always be used to group the vdatas in a hierarchical manner.
Vdatas that are related should be grouped together. Since it is
possible for several vdatas to point to one vgroup or vdata, a well-
chosen grouping saves on file storage by avoiding duplication of
data. Figure 1.4 shows an HDF file containing three vsets. The
three vsets have one vdata in common.

Figure 1.3    Vset Schemes








Figure 1.4    HDF File with Three Vsets





In this file, three vsets exist, each consisting of one vgroup
referencing two vdatas. The vgroups are named "result 1,"
"result 2," and "result 3," respectively. Since the positional
values do not change during the simulation, only one vdata,
defined to contain the fields "PX, PY," is needed. The three vsets
then "share" this vdata. On the other hand, the temperature values,
different for each time, are stored as three separate vdatas, each
defined to contain the field "TMP" for temperature.


Examining the HDF Vset Interface

The HDF Vset interface is a library of routines for the definition,
organization, and manipulation of vgroups and vdatas, as well as
for data access. The routines are classified according to the action
they perform:

Table 1.1    Vset Routines
Ñ    access    grants or terminates access to vgroups or vdatas.
Ñ    inquiry    returns information about vgroups or vdatas.
Ñ    search    searches and locates specific vgroups or vdatas.
Ñ    specify    sets up specification for data access.
Ñ    read/write    retrieves/stores data from/in vdatas.
Ñ    linkage    links vgroups and vdatas to form vsets.


Using Examples

The following examples demonstrate the ease in reading and
writing data using the vset interface routines. (See Appendix D
"Source Files," on compiling and creating executables.)


Example 1:
Writing Data
Figure 1.5 shows a simple program that stores data as a vset in an
HDF file. It takes pxy, a 2D array (float) of 1000 points (x, y
values), and pv, an array (float) of 1000 temperature values, and
stores them as two vdatas linked to a vgroup (vset A in Figure 1.3).
One vdata is defined to contain one field, "TMP," for temperature,
while the other is defined to contain two fields, "PX, PY," for the
positional values. The vgroup is named "mypoints."

Figure 1.5    Storing Vset Data

    #include "vg.h"

    main()     {

        VGROUP        *vg;
        VDATA        *vs;
        DF        *f;
        float        pxy[1000][2], temp[1000]; /* (x,y) temperature values */


        f = DFopen("myfile.hdf", DFACC_ALL,0); /* open HDF file with full access */

        /* create a vgroup with write access, then name it "mypoints" */
            vg = (VGROUP*) Vattach(f, -1,"w");
            Vsetname(vg,"mypoints");

        /* create a vdata to store x,y values */
            vs = (VDATA*) VSattach(f,  -1, "w");
            VSsetfields(vs, "PX,PY");
            VSwrite(vs, pxy, 1000, FULL_INTERLACE);
            Vinsert(vg, vs);
            VSdetach(vs);

Figure 1.5    Storing Vset Data (Continued)

        /* create a vdata to store temperature values */
            vs = (VDATA*) VSattach(f, -1, "w");
            VSfdefine (vs, "TMP",  LOCAL_FLOATTYPE,1);
            VSsetfields(vs, "TMP");
            VSwrite(vs, tmp, 1000, FULL_INTERLACE);
            Vinsert(vg, vs);
            VSdetach(vs);

        Vdetach(vg);
        DFclose(f);
        }


The program in Figure 1.5 does the following:

1.    The HDF file is opened.

2.    A new vgroup is created by calling Vattach with an id
argument of -1. This vgroup is then named ╥mypoint.╙

3.    A new vdata is created by VSattach with an id of -1.

4.    VSsetfields specifies that the vdata will contain two float
fields "PX, PY".

5.    VSwrite writes out the data as 1,000 pairs of x, y values.

6.    Vinsert establishes a link from the vgroup "mypoints" to this
vdata.

7.    Access to the vdata is terminated with VSdetach.

8.    A similar sequence of calls creates a second vdata, specifies
that it contains the field "TMP," writes out the temperature
values, links the same vgroup to the vdata, and detaches the
vdata.

9.    Finally the vgroup itself is detached, and the HDF file is closed.

The fieldnames "PX" and "PY" are names pre-defined by the
interface to represent x and y float values. However, the fieldname
"TMP" is not pre-defined and it must first be defined with a
VSfdefine call specifying that "TMP" is a field of one float value.
(See Appendix C "Pre-defined Fieldnames," for a list of all pre-
defined fieldnames.)

Each call to Vattach and VSattach returns a pointer to a vgroup
and vdata, respectively. The other routines then manipulate the
vgroups and vdatas through these pointers.

The fourth argument to VSwrite is the buffer interlace, a
parameter that tells VSwrite how data in the buffer is laid out. A
value of NO_INTERLACE specifies that all the data values for one
field are contiguous, while a value of FULL_INTERLACE specifies
that one data value of a field is immediately followed by one data
value of another field.
The vdata in the file also has its own interlace, called file
interlace, which equals FULL_INTERLACE by default. Chapter 2,
"Vdatas," describes buffer and file interlacing in detail.


Example 2:  Reading Data
The program in Figure 1.6 reads the data from the HDF file
created by the program in Figure 1.5. Assume it is known that a
vgroup exists named "mypoints" in the HDF file, and that this
vgroup contains (x, y) coordinate values in its first vdata and
temperature values in its second vdata; but, it is not known how
many vertices exist (vset A in Figure 1.3).

Figure 1.6    Reading Data from a Vset

    #include "vg.h"
    main() {

        VGROUP        *vg;
        VDATA         *vs;
        DF        *f;
        float        (*pxy)[2]; /* pointer to an array of 2 floats  (x,y coords) */
        float        *temp;     /* pointer to an array of floats  (temperature ) */
        int        nvertices, vsize, interlace;
        char        vsname[50], vgname[50], fields[50];
        int        vsid, vgid;


        f = DFopen("myfile.hdf", DFACC_ALL,0); /* open HDF file with full access */


        /* first , locate that vgroup. set vgid to -1 to begin search */
            vgid = -1;
            while ((vgid=Vgetid(f, vgid)) != -1) {
                vg = Vattach(f, vgid,"r");     /* attach with read access */
                Vgetname(vg, vgname);
                if (!strcmp(vgname,"mypoints"))
                    {  found = 1;  break; }    /* found the vgroup */
                Vdetach(vg);
                }
            if (found == 0) perror ("did not find vgroup. exit");
        /*  next, extract the (x,y coords) from the  1st vdata */
            vsid = -1;
            vsid = Vgetnext(vg, vsid);
            vs = (VDATA*) VSattach(f, vsid, "r");
            VSinquire(vs, &nvertices, &interlace, fields, &vsize, vsname);
            pxy = (float*) malloc(nvertices * sizeof(float) * 2);
            VSsetfields(vs,"PX,PY");
            VSread(vs, pxy,nvertices, FULL_INTERLACE);
            VSdetach(vs);

        /*  finally extract the temperature values from the 2nd vdata */
            vsid = Vgetnext(vg, vs id);
            vs = (VDATA*) VSattach(f, vsid, "r");
            VSinquire(vs, &nvertices, &interlace, fields, &vsize, vsname);
            temp  = (float*) malloc(nvertices  * sizeof(float) );

            Vsetfields(vs,"TMP");
            VSread(vs, temp, nvertices, FULL_INTERLACE);
            VSdetach(vs);
        Vdetach(vg);
Figure 1.6    Reading Data from a Vset (Continued)

        DFclose(f);
         ...
        free(pxy);
        free(temp);

        }


The above program illustrates the following:  (1) searching for a
specific vgroup, (2) performing inquiries, and (3) determining the
amount of memory space to allocate.

The program does the following:

1.    The HDF file is first opened.

2.    It searches the HDF file for a vgroup named "mypoints" by
sequencing through all vgroups in a loop. The search is
initiated by setting the vgid argument to -1 in the call to Vgetid.

3.    The call returns an id of a vgroup or -1 if none exists.
Whenever a vgroup id is returned, an attach is done to that
vgroup, and its name, as returned by Vgetname, is checked. A
flag is set if the vgroup is found.

4.    Once the vgroup is found, its first vdata is attached. The call to
VSattach requires an id, which is obtained by a call to
Vgetnext. Vgetnext returns the id of the first vdata, if its id
argument is -1.

5.    VSinquire returns, among other information, the number of
elements stored in that vdata. This number multiplied by the
size of an element (two floats) gives the size of the allocated
space.

6.    VSsetfields then specifies that the data fields "PX, PY" in that
vdata will be accessed.

7.    VSread reads the data into the allocated space, pxy.

8.    The vdata is detached when reading is completed.

9.    A similar sequence of calls extracts the temperature values
from the second vdata into the allocated space, temp. Note the
use of Vgetnext, which is being passed the id of the first vdata.
This call returns the id of the next entity after the first vdata.


Example 3:  Using Meshes
and Connectivity Lists
Many scientific and graphic applications use meshes to describe
surfaces and objects as well as their properties. The shape of the
surfaces and objects are defined as a set of connected points or
nodes. The properties of the mesh are values attached to each node
or group of nodes within the mesh.

The HDF Vset allows the mesh description to be stored together with
its associated values. One way to represent and store meshes in an
HDF file is to define a vertex-set. A vertex-set is an instance of an
HDF Vset that, minimally, consists of the three following
component datasets:

1.    A set containing the coordinates of the nodes in the mesh

2.    A set that describes how the nodes are connected to each other to
form the shape of the mesh

3.    One or more sets of data values that correspond to each of the
nodes or polygons

The second dataset is a matrix called a connectivity list. Each
element in a connectivity list describes a polygon within the mesh.
Each polygon is a list of all the nodes that form the polygon. The
connectivity list, then, is a set of polygons that form the mesh. Each
of the above component sets may be stored as a separate vdata.
These vdatas should then be grouped together with other descriptive
or annotative information under a vgroup.

As an example, assume that the 1000 nodes in Figure 1.3 actually
form a mesh of 400 triangles (Figure 1.7). (Each node has a node
number in addition to its coordinate position.) The connectivity of
the mesh is described by an array of node numbers, in the array
mesh. Every three numbers in one row of the table define a
triangle. This information can be stored as a connectivity list in
another vdata, as the user-defined field "PLIST" (polygon list).
The data can then be related to the position and temperature values
by linking the vdata to the existing vgroup (of vset A in Figure 1.3).

Figure 1.7    Mesh and Connectivity List





The program in Figure 1.5 is easily modified to write a vertex-set
into an HDF file by adding code to store the connectivity list in the
same vset.

The following code segment (Figure 1.8) stores the connectivity
list (i.e., the array mesh) into the vset A in Figure 1.3, thereby
creating the mesh vset as shown in Figure 1.7.

Figure 1.8    Storing a Connectivity List

        int mesh[400][3];
            .....
        /* create a vdata to store mesh  */
            vs = (VDATA*) VSattach(f,  -1, "w");
            VSfdefine (vs, "PLIST",  LOCAL_INTTYPE, 3);
            VSsetfields(vs, "PLIST");
            VSwrite(vs, mesh, 400, FULL_INTERLACE);
            Vinsert(vg, vs);
            VSdetach(vs);
            .....


This code segment should be inserted just before the Vdetach (vg)
statement in the program in Figure 1.5.

The connectivity list is stored in a new vdata that is then linked to
the vgroup of vset A. Creating and storing connectivity
information to a vdata is no different than the storing position and
temperature values in the program. (See the outlined steps below.)

1.    The code creates a new vdata.

2.    VSfdefine defines "PLIST" as one field comprising three
integers. This step is necessary because "PLIST" is not a pre-
defined fieldname. VSsetfields then sets up the vdata to
contain this field.

3.    VSwrite writes out the connectivity data for 400 triangles from
the array mesh into the vdata.

4.    Vinsert links this vdata to the same vgroup.

5.    VSdetach terminates access to this vdata. The resulting
configuration is shown in Figure 1.7.

Note that "PLIST" is a single field comprising three integers;
hence, the call to VSfdefine with the fourth argument equal to 3.
This argument specifies the order of the field. "PLIST", as defined
for this vertex-set, is a single field of order 3. Although one
"PLIST" element comprises three integers, these three integer
values can only be accessed together. Thus the following code
segment writes out three integer values; i.e.,  mesh[0][0],
mesh[0][1] and mesh[0][2]. (The numbers of the three nodes
form the first mesh triangle.)

        VSsetfields(vs, "PLIST");
        VSwrite (vs, mesh, 1, interlace);


Similarly in the code segment below, the call to read one ╥PLIST╙
element from the vdata returns three integers (the numbers of the
three nodes forming this first mesh triangle) in the array x:

        int x[3];
        ...
        VSsetfields(vs, "PLIST");
        VSread (vs, x, 1, interlace);


Example 4:  Linking
Storage Elements in Vsets
The program segment in Figure 1.9 illustrates the use of the
routine Vinsert. This routine allows several vgroups to refer to
one vdata in the HDF file, and hence, "share" data within the
vdata.

The dataset comprises pxy, a 2D array (float) of 1000 points
(x, y coordinates) and  three arrays t1, t2, and t3, each an array
(float) of 1000 temperature values taken from a simulation at three
different times. The points remain constant at all times. This
dataset is best represented  by three vsets, each representing the
simulation at a different time (vset B in Figure 1.3). Thus, four
vdatas exist, one for each temperature dataset and the fourth for the
points (x, y coordinates).

Figure 1.9    Linking Vgroups
and Vdatas

    #include "vg.h"

    main() {
        VGROUP              *vg1, *vg2, *vg3;
        VDATA             *vs;
        DF                     *f;
        int             b,n;

        float           pxy[1000][2];                    /* (x,y) values */
        float           t1[1000], t2[1000], t3[1000]; /* temperature values */

        /* open HDF file */
                    f = (DF*) DFopen("myfile.hdf", DFACC_ALL,0);

        /* attach a 3 new  vgroups with write access */
            vg1 = (VGROUP*) Vattach(f, -1,"w");
                    Vsetname(vg1, "result1");            /* name it "result1" (optional)*/
                vg2 = (VGROUP*) Vattach(f, -1,"w");
            Vsetname(vg2, "result2");            /* name it "result2" (optional)*/
                vg3 = (VGROUP*) Vattach(f, -1,"w");
                Vsetname(vg3, "result3");          /* name it "result3" (optional)*/

        /* attach a new vdata to store temperature values t1. Insert into vg1. */
                vs = (VDATA*) VSattach(f, -1, "w");
               b = VSfdefine (vs, "TMP",  LOCAL_FLOATTYPE, 1);
                b = VSsetfields(vs,"TMP");
                n = VSwrite(vs, t1, 1000, FULL_INTERLACE);
             Vinsert(vg1, vs);
Figure 1.9    Linking Vgroups
and Vdatas
(Continued)

               VSdetach(vs);

        /* attach a new vdata to store temperature values t2. Insert into vg2. */
                vs = (VDATA*) VSattach(f, -1, "w");
                b = VSfdefine (vs, "TMP",  LOCAL_FLOATTYPE, 1);
                b = VSsetfields(vs,"TMP");
                n = VSwrite(vs, t2,1000, FULL_INTERLACE);
            Vinsert(vg2, vs);
               VSdetach(vs);

        /* attach a new vdata to store temperature values t3. Insert into vg3. */
                vs = (VDATA*) VSattach(f, -1, "w");
                b = VSfdefine (vs, "TMP",  LOCAL_FLOATTYPE, 1);
               b = VSsetfields(vs,"TMP");
               n = VSwrite(vs, t3,1000, FULL_INTERLACE);
            Vinsert(vg3, vs);
               VSdetach(vs);

        /* attach a new vdata to store (x,y) values */
        /*This vdata will be shared by the three vgroups */
                vs = (VDATA*) VSattach(f, -1, "w");
                b = VSsetfields(vs,"PX,PY");
               n = VSwrite(vs, pxy,1000, FULL_INTERLACE);

        /* Insert  vs into vgroups vg1, vg2, and vg3, then detach it. */
                b = Vinsert(vg1, vs);
               b = Vinsert(vg2, vs);
               b = Vinsert(vg3, vs);
            VSdetach(vs);

        /* done, detach all vgroups */
                 Vdetach(vg1);
            Vdetach(vg2);
                 Vdetach(vg3);

               DFclose(f);
        }


The above code segment does the following:

1.    The HDF file is opened.

2.    Three new vgroups are created and attached.

3.    Three separate vdatas are created and attached. Separate sets of
temperature data from t1, t2, and t3 are then written to each
vdata.

4.    Each vdata is then linked to the respective vgroups using
Vinsert. After this step, the three vdatas are then detached.

5.    A fourth vdata is created and attached for storing positional
data. Immediately after data is written to it, the three calls to
Vinsert each create a link from each of the three vgroups to this
vdata. The vdata is then detached.
6.    All vgroups are  detached and the file is closed.

After the insertions, each vgroup logically owns two vdatas. Note
that the shared vdata may be accessed by accessing any of the
vgroups╤the vdata now belongs equally to all three vgroups.

Vinsert may also be used to insert one vgroup into another vgroup
in a similar manner.


Using the HDF Vset Routines

All the vset routines require that the HDF file be opened using
DFopen with the access-mode DFACC_ALL. The routines provide six
basic functions:  search, access, inquiry, specify,
read/write, and linkage:

Ñ    Search routines provide a systematic way of searching through
the HDF file for vgroups and vdatas.

Ñ    Access routines attach, or grant access to, vgroups or vdatas
before any data transfer can occur. They also detach, or
properly terminate access to, vgroups or vdatas when data
transfer is completed.

Ñ    Inquiry routines return information about existing vgroups or
vdatas. They are useful for searching through the file for a
specific vgroup or vdata, and for determining the fields and
data sizes of vdatas.

Ñ    Specify routines define and set up parameters that dictate the
format and amount of data to be read or written. They are also
used to define new fields and to assign names to vgroups and
vdatas.

Ñ    Read/Write routines perform the actual data transfer between
the file and the calling program. They are always preceded by
the appropriate access and specify routines.

Ñ    Linkage routines link vgroups with vdatas or other vgroups into
a hierarchy that is the vset.

The examples in this chapter are typical of the order for calling the
routines. In most cases, you will use the routines in the following
order:

Ñ    Search for a vgroup.

Ñ    Attach a vgroup, a procedure that returns a pointer  to a vgroup.

Ñ    Sequence through that vgroup and search for the desired vdata.
Vgroups that are attached to the vgroup should be similarly
searched.
Ñ    Attach the desired vdata when found.

Ñ    Execute inquiry calls at any time, to get information about
existing vgroups or vdatas (e.g., its name, fieldnames, element
size, interlace, etc).

Ñ    Perform a series of specify calls on the vdatas to establish the
fields to be accessed and the type of interlace used. The specify
calls must be performed before any data transfer can occur. For
more information on interlace, see Chapter 2, ╥Vdatas.╙

Ñ    Execute a series of read/write calls to read or write data.
Ñ    Link newly created vgroups or vdata into new or existing
vgroups or vdatas.

Ñ    Perform a detach to terminate access to every vdata or vgroup
that was attached when linkage is completed.

Ñ    Use the access routine VSseek, if necessary, for random-access
reads within a vdata. (NOTE:  Random-access writes are
forbidden.)

Ñ    Close the HDF file by calling DFclose.


Naming Vgroups and Vdatas

When there is more than one vgroup or vdata within a file, a
problem arises:  How can one uniquely identify or refer to a
particular vgroup or vdata? One way to solve the problem is to
always name each vgroup or vdata. (Using Vsetname and
VSsetname). Assigning a unique name enables you to easily
identify vgroups or vdatas in the future. The routines Vgetname
and VSgetname each return the name of a vgroup or vdata,
respectively. Thus, rather than guessing which vgroup or vdata to
use, you use its name for searching.

Naming vgroups or vdatas is optional. Vgroups or vdatas may also
be located if their identifiers are known. The identifiers are
unique, and are actually the reference component of their HDF
tag-ref identifiers. They are values used and returned by the
search routines (Vgetid, VSgetid, Vgetnext).

1.    Pass an identifier of -1 to the search routine. The routine then
returns the identifier of the first vgroup or vdata.

2.    Pass the returned identifier back to the routine. This will then
return the identifier of the next vgroup or vdata. No more
vgroups or vdatas are found if the routine returns -1.

3.    Use the valid returned identifier in an attach call in order to
access that vgroup or vdata.

Within a vgroup, there may be vdatas and other vgroups. The
identifier itself does not identify if an entity is a vdata or vgroup.
The inquiry routines Visvg and Visvs respectively test if an id
refers to a vgroup or vdata, respectively. Searching is illustrated
in Figure 1.6 in the section example, ╥Reading Data.╙